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Preface 



This report is based on a study conducted by the Consortium of Professional 
Associations for the Study of Special Teacher Improvement Programs (COMPASS) for the 
U.S. Office of Education. 

The Consortium was formed in May, 1966, by the American Historical Association, 
the Association of American Geographers, the Department of Audiovisual Instruction 
(NEA), the International Reading Association, and the Modern Language Association of 
America. Invitations were later extended to, and accepted by, the American Economic 
Association, the American Industrial Arts Association, and the American Political Science 
Association. Four members at large provide liaison with the arts and humanities, psychological 
tests and measurement, educatioml psychology, and teacher education specialists. 

The objectives of COMPASS are to provide a coordinated assessment of the 
effectiveness and impacts of institutes and other special teacher-training programs; to 
propose means of improving such programs; and to provide a medium for dialogue among 
the professional associations and leading scholars of the several subject content disciplines 
and fields represented on its Board. In the past, the Consortium has conducted studies of 
summer institutes in individual disciplines; it is presently sponsoring an extensive study of 
the impact of summer institutes in four disciplines upon participants in the institutes. 

The present study of the Experienced Teacher Fellowship Program was contracted by 
COMPASS to Clark University, to be conducted under the supervision of the Consortium 
Board. The research was initiated by Professors Crockett and Bentley, Professor Laird 
participated in the analysis of the results and in the writing of the report. The research 
staff spent four days in a writing conference in July, 1967, with Drs. John Thompson, 

Saul Cohen, William Engbretson, Richard Longaker, and Mr. John Cogan; at this conference, 
the results were studied in detail and the outline of the present report was formulated. 
Preliminary drafts of the report were examined by the members of the writing conference r 'd 
by the Executive Committee of the Consortium; the final version of the report has benefitted 
extensively from their comments. 

This edition of the report has been prepared in October, 1967, for advance circulation. 
A later, bound edition will be issued by the Consortium Office. 
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I. Introduction 



The Experienced Teacher Fellowship Program is a unique and imaginative venture. 

Its ultimate objective, and that of two other, closely related programs, is to improve 
the quality of education in the nation's elementary and secondary schools. The three 
programs pursue this objective in two ways: by assisting selected, potentially influential 
teachers to pursue full-time graduate education in specially planned courses of study, 
and by fostering and strengthening an increased concern for the training of teachers. The 
Experienced Teacher Fellowship Program sponsors special programs that provide financial 
support for graduate studies to teachers with field experience. A second program, the 
Prospective Teacher Fellowship Program, supports similar kinds of programs for individuals 
who have no teaching experience but who expect to become elementary or secondary 
school teachers. The third program, the Institutional Assistance Grant Program, awards 
financial grants to strengthen the graduate programs for teacher preparation in 
institutions that have already been awarded either an Experienced or a Prospective Teacher 
Fellowship Program. The present report summarizes a preliminary study of the Experienced 
Teacher Fellowship Program. 

A. The Development of the Experienced Teacher Fellowship Program. 

The history of the ExTFP belies the generalization that governmental programs 
develop slowly. It was authorized under Title V, Part C of the Higher Education Act of 
1965. Guidelines for the program were distributed in two letters, dated December 27, 1965, 
and January 10, 1966. The deadline for mailing completed proposals was January 20; a 
panel of consultants read, evaluated, and rated the proposals in the period from January 24 
to 26; and the announcement of awards was made in February, barely two months after the 
first guideline was sent out. The first students began their study in June, 1966. 

Despite the speed with which the program was mounted, almost 1,000 proposals 
were submitted for the academic year 1966-67. Fifty of these proposals were funded, 
enabling just over a thousand experienced teachers, from all parts of the country and representing 
diverse disciplines, to spend a year (in a few programs, two years) in fulhtime graduate study. 

In its underlying assumptions, the conception of the ExTFP was broad and inclusive. 

In the guidelines, no limits were suggested as to the range of subject matter that would 
be supported; no premium was placed on either innovation or traditionalism in educational 
procedures, and there was no attempt to specify in detail the structure that the graduate 
programs should adopt. There was, however, the assumption that graduate education is 
most effective when the courses a student takes are related to one another in a meaningful 
fashion. The guidelines for ExTFP proposals incorporated this assumption by setting three 
restrictions on authors of proposals: 

First, evidence was required of more than perfunctory cooperation between subject- 
matter and teacher-education specialists. All proposals were required to demonstrate that 
a suitable faculty could be arranged for, composed of members of "teacher education" and 



i “non-teacher education" departments hurther, both the chairman or dean for the substantive 
aspect of the program and the chairman or dean for teacher education were required to sign 
the proposal before it was submitted. 

Second, institutions were required to adopt an en bloc procedure, by designing 
; a program for the entire group of fifteen to thirty fellows, rather than leaving the individual 
^ fellows "to the mercy of the catalogue's cafeteria- like offerings, so often unsuited to the 
needs of experienced personnel . " The en bloc mode of organization was also to provide 
greater visibility of the program on the campus as well as increased opportunity for fellows 
to profit from interaction with their peers and from formal instruction by their professors. 

Third, the guidelines encouraged cooperation between the institution of higher 
education and the local school district or system. This was fostered in part by the require- 
s ment that fellows be selected jointly by their home educational system and by the college 
^ or university concerned. School administrators were required to recommend applicants, 
and applicants were expected to return to the school systems from which they came. In 
I addition, in order to confront the realities of teaching in schools, cooperation was encouraged 
; between colleges and local school systems, to provide a meaningful practicum experience 
; for the participants. 

The fifty programs that were funded were held in forty-seven different colleges and 
I universities. Programs were conducted in 17 different disciplines, ranging from general fields 
I of education (elementary education, teaching the disadvantaged, and counseling and guidance) 

‘ through the traditional liberal arts disciplines, and including specialized areas 'uch as 
health education, the school library, and educational media. The fellows were drawn from 
: every part of the country and from schools which served every economic level; their educational 
: assignments ranged from preschool to high school. 



B . Evaluation of the Experienced Teacher Fellowship Program 



1 . Three Projected Evaluation Studies 



Just as the ExTFP was planned and instituted with considerable speed, so, also, were the 
I procedures for studying the program's effectiveness . Barely three months elapsed between 
the formation of a research team and completion of data collection for the present report. 

During that time, a plan has taken form which foresees a series of three related investigations 
! of the effectiveness of the ExTFP: a questionnaire study of responses to the first yearns 
program; during the second year, a field investigation of the operation of the ExTFP in three 
different institutions; and, in the third year, another study of the entire set of institutions 
; then involved in the ExTFP. Each successive investigation will build on the results obtained 
by those preceding. 



The first of these studies, based on questionnaires and visits by teams to selected 
programs, will be described at length below. The intensive pilot study of three individual 
programs will be carried out during the academic year 1967“68; it will involve repeated 
interviews with participants and faculty in each institution and the periodic administration 
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of questionnaires and other tests. The extensive body of information that will be obtained 
will permit an acquaintance In depth with the operation of these three Institutions, making 
it possible to identify factors that appear to account for the effects the programs have upon 
the fellows, the faculty, and the institution. An important aspect of this second study will 
be interviews with the fellows after they have returned to their home schools in 1968-69. 
The third investigation, to be initiated during the academic year 1968-69, will be an 
extensive study of all the Experienced Teacher Fellowship Programs then in operation, 
using self-report measures, interviews, and observations, all developed out of earlier 
research experiences. 

2. The Procedures Used in the Present Investigation 

I — 

The present report rests upon two kinds of data: responses to questionnaires 
that were administered to the individuals who were actually involved in the program, and 
reports by teams of evaluators who visited 31 of the 50 programs. 

Four questionnaires were constructed for administration to those involved in the 
programs. Each questionnaire borrowed heavily from those used in earlier studies of 
summer institutes. One questionnaire, containing some 60 different items, was administred 
to the fellows at the institutions they attended, under conditions which assured anonymity . 
Completed questionnaires were obtained from 940 of the 1,004 fellows, representing 49 of 
the 50 institutions.^ 

The director at each institution was asked to supply the names of the full-time 
and part-time staff of his program. A copy of a second questionnaire, about equal in 
length to the student questionnaire, was then mailed to every full-time faculty member 
and to five randomly- selected part-time facultv members on each campus. Completed 
questionnaires were obtained from 187 faculty members, in 47 different institutions. 



A third questionnaire, sent to the director at each institution, was identical to 
the faculty questionnaire except that it contained an additional set of ten items concerning 
the administration of the program. Of the 50 directors, 45 returned these questionnaires 
in time for analysis in the present report. 

The fourth questionnaire, intended to assess the impact of the program upon the 
existing teacher-education procedures at the institutions, was sent to the director of teacher 
education on each campus. Response to this questionnaire was spotty; for this reason, these 
replies will not be discussed in detail in this report. 



^Questionnaires were not received from the program in Social Studies at the Universi 
of Minnesota. 



The evaluation teams, which visited 31 of the 50 programs, normally consisted or 
three persons: o specialist in the subject matter of the institute, a specialist in teacher 
education, and a teacher experienced in the relevant subject matter . Some 85 
of these evaluation teams met in early April with the research team, members of CONPASS, and 
representatives from the Office of Education for a discuaion of the e/aluation rating scale 
ai^ of the procedure that was to be followed in the evaluation visit. They then ^ 

days on the campuses to which they were assigned, meeting with faculty, students, 
administrators, visiting classes, and reviewing the general 

Subsequently, each team member individually completed a Visitors Evaluation Form, 
conta?ning 24 different items. For each item, the evaluator rated the program on a 7-point 
scale, and then was asked to provide a written analysis of that aspect of the pr^ram s 
operation in explanation of his rating. In addition to the individual reports, the team 
members submitted a combined evaluation on each item of the evaluation form; this lost 
report represented the consensus of all the team members. 






I 
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'Because of difficulties in scheduling members of evaluation teams, three institutions 
were visited by teams of only two members; at two institutions, the team contained four 

members. 
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II. An Overview of the Results 



In the following chapters a detailed report will be made of the results of this study. 

The purpose of the present chapter is to point out the highlights of these results. 

For the most part, fellows who took part in the Program were relatively young 
teachers, but experienced ones. Their ability and motivation, apparently, was extremely 
high; according to program directors and faculty members, the fellows were at least equal, 
if not superior, in quality and motivation to the regular graduate students at the institutions 
concerned. The educational attainments and experience of faculty members suggests that 
the average teacher in the Program, olso, was more than adequately qualified. Thus, the 
great majority of the fifty programs possessed the two principal qualifications for an effective 
academic program: an able, highly motivated student body and a capable, concerned faculty. 

The extent to which the Program's potential effectiveness was realized and the 
general correlates of effectiveness may conveniently be summarized by six briwd generalizations 
for the specific results on which these generalizations are based, the reader should consult 
the body of the report . 

1 . The reaction to the P rogram by program oirectors/ faculty members ^ fellows^ 
and evaluation teams was overwhelmingly favorable. 

As a general rule, the extent of a source's enthusiasm about the Program varied 
with that source's degree of professional investment in it: directors' responses were usually more 
favorable than those by faculty members, faculty members were more favorable than fellows, 
and fellows more favorable than evaluators. But this general rule held within a context 
of over-all favorableness toward the program. Specific evidence of the widespread 
approval that was generated may be found throughout the results. The extent of this 
approval may be illustrated by the fact that 82% of the fellows reported that their own 
program was either usually stimulating and interesting or stimulating and interesting throughout. 
Responses by faculty members and program directors to the identical itern were even more 
favorable. Similarly, the majority of respondents in each of the four roles -- directors, 
faculty members, fellows, and evaluators - reported that the Program had clearly met the 
educational needs of the fellows. Even the few evaluation teams which were sharply critical 
of an individual program took care to comment favorably on the over-all concept of the txIhP. 

There was, of course, a considerable variation among institutions in the evaluations 
that were received: some programs were given extremely high ratings, a few received 
relatively low evaluations. It must be stressed again, however, that this variation took 
place around an average value that was very favorable, indeed. 
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2 . Ther e was a high degree of solidarity and morale among fellows in the averag^ 
irogramTthe' level of solictority arid morale correlated positivdy with judgments of effectiveness. 

Again there was considerable variation in the level of morale and solidarity that 
sharacterized the different institutions; nevertheless, the average program received high 
atings on morale and solidarity from directors, faculty members, fellows, and evaluators alike, 
ieyond this, there were consistent positive correlations between estimates of solidarity and 
norale and judgments of the effectiveness of individual programs: programs where morale 
ind solidarity were high also received high ratings on effectiveness and participant satisfaction; 
^^hen morale and solidarity were low, so were ratings of satisfaction and effectiveness. 

3. The amount of work assigned was heavy, and inversely r elated to effectiveness^ 
and satisfaction. 



Of the 31 evaluation teams, none said the fellows' work load was too light, while 
21 said it was in some degree too heavy; of 940 fellows, only 6 said the load was too light 
bnd 440 said it was "too heavy to allow completion of assignments and independent work. 

Of particular interest was the inverse relationship between fellows' and evaluators 
judgments on this question and the various measures of satisfaction and effectiveness: 
institutions where the amount of work required was judged to be inordmately high were 
consistently ranked as relatively ineffective. It should be noted that this relationship held 
Ifor the absolute amount of work that was required, not for the amount of competitiveness that was 
fostered between fellows. In the average program, fellows reported a fairly large amount 
of competition with one another; however, these latter ratings did not relate consistent y 
either to judgments of the over-all work load or to ratings of effectiveness and satisfaction. 

4. Res pondents in different roles disagreed as to whether the pro grams built on the 
extensive~bdckgrounds of fellows; judgments by fellows and evaluators on tf^q uest^ 

I correlated positively with measures of effectiveness. 



Almost all of the program directors and a large majority of the faculty reported 
that the curriculum at their institution utilized and built upon the experience of the fellows; 
most of the evaluation teams indicated the reverse; answers by the fellows to this question were 
: intermediate, but more similar to the faculty's than to the evaluators'. Despite their 
; disagreement in the level at which they felt the fellows' experience was utilized, evaluation 
teams and fellows agreed in their rankings of institutions on their achievement of this gew . 

' Furthermore, those programs which, according to fellows and evaluators, managed somehow 
to build upon the fellows' experience received more favorable ratings on program effectiveness 

than those which did not. 

5 Respondents in different roles disagreed as to the extent of co operation among 
; and the am ount of innovation in the programs; depo rtmental cooperation, but 

! not innovation was correlated with program effectiveness. 



For the most part, directors and faculty members involved in the Program reported that 
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cooperation among different departments was good, that the Program had considerable 
effects on teacher-education procedures, and that it contributed to the plans for development 
of the department and institution concerned. Evaluation teams did not make such favorable 
judgments. Although in some institutions evaluators said that the Program had affected 
interdepartmental cooperation, teacher training, or departmental development, in as many 
other institutions evaluators felt that it had little effect on such policies. It seems likely 
that the evaluators' judgments were somewhat closer to reality than those of the directors 
and faculty; that within the few months of the Program's operation it had effected few, if any, 
really substantial changes in the structure of most of the host institutions. It is worth 
noting, however, that there was a positive correlation between judgments of program 
effectiveness on the one hand and, on the other, reports by evaluators and by fellows of 
effective interdepartmental cooperation. 

As with judgments of effects upon host institutions, program directors and faculty 
members were much more likely than evaluators to report that their programs contained 
imaginative innovations. Again it is likely that the evaluators' judgments were more 
objective than those by the other two groups, and that as many programs introduced few 
educational innovations as introduced many. It is noteworth that evaluator estimates of 
the extent of innovations was uncorrelated with judgments of program effectiveness. 

As adequate a summary as any of the effects of the Program upon the host institution 
can be given by quoting from an interim report on this project, written in May, 1967: 

The Programs appear to have been least effective in overcoming 
the traditional patterns of organization in colleges and universities. 

Thus, the most common complaints [by evaluation team^ dealt with 
the similarity of these programs to traditional undergraduate and 
graduate education, the imposition of a common body of required 
courses upon all participants, the failure to adjust the curriculum to 
the needs of individual students, the absence of true collaboration 
l^etween different departments of the same institution, or the unconcern 
of the staff for the response to the Program of the fellows as a group. 

All of these complaints are commonly voiced throughout higher education 
in America; they are not unique to the Experienced Teachers Fellowship 
Program. It is significant that a considerable number of institutions 
were adjudged to provide for their fellows an unusual and rewarding 
educational experience, some by following traditional educational 
patterns, other by breaking with tradition and establishing novel and 
exciting educational procedures. 

6. The program director has an extremely important role in determining the 
effe ctiveness of individual programs. 

A partial enumeration of the functions that a program director performs yields a list 
of impressive length. He should be directly involved in deciding upon the course content and 
the mode of organization of the program, he must make sure that the formal courses and 



the supplemental activities are coordinated, must arrange for the presence of vrhatever 
educational materials are required, must encourage informal exchanges among fellovrs and 
between fellows and staff, must try to mediate in disputes that may develop among 
participants, must ascertain the fellows' and the faculty's views and criticisms of the pr^ram 
and its effectiveness, must decide whether changes in procedures or content are required, and, 
when the decision is affirmative, must determine what chaises to make in the program 
and how to make them. In a program whose success relies in good part upon the 
establishment of high esprit de corps among participants and upon the group s perfoi^nce 
en bloc, the fulfillment of these functions can be critically important. There are doubtless 
some programs which run smoothly from beginning to end, never requiring the mediating 
influence of a skillful administrator. In the typical institution, however, at some time 
during the year crises arise, interests conflict, difficulties occur which require effective 
administrative action. At such rimes it is essential that the program director possess the 
ability, the time, and especially, the institutional power to respond effectively to the demands 

of the situation. 
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III. The Popu lotion and the Programs 



We have already remarked that the ExTFP embraced a wide variety of offerings. 

Programs varied not only in the type of institutions that were involved, in the geographic 
region where the institutions were located, in the characteristics of fellows they enrolled, 
and in the subject matter that they offered; beyond this, they differed remarkably in the 
pedagogical strategy that they adopted: some were innovative, others traditional; some 
strove for competitiveness among fellows, other sought a non~ competitive atmosphere; 
some established informal relationships between fellows and faculty, others maintained 
formal relationships. The programs and the individuals involved in them differed in these 
general ways and in all other ways in which people differ. Our purpose in this section 
of the report is to summarize some of the characteristics of the fellows, the faculty, and 
the programs* In succeeding sections, we shall discuss reactions to these programs, and 
shall look for variables that correlated with their effectiveness. It may sometimes appear 
that the differences among programs are obscured in the course of this analysis, that diversity 
is reduced to uniformity. If so, the reader should bear in mind that we are seeking for 
whatever underlying constancy there may be beneath the remarkable surface diversity. 

A. Characteristics of the Participants 

1 . Personal Characteristics 

In certain of their personal characteristics the group of fellows was not entirely 
representative of teachers as a whole. Men made up 51% of the group, no doubt a higher 
proportion than obtains among teachers in general. In addition, the group was relatively young, 
with 79% being younger than 40 and 28% younger than 30. Despite their relative youth, the 
participants were not inexperienced in teaching. Ninety-two percent reported three or 
more years of experience in education; 59% had six or more years of experience; however, 
only 24% had ten or more years of experience. The participants' experience spanned all 
levels of elementary and secondary education: 32% had been principally involved at the 
high school level, 21% at the junior high school level, 45% at the elementary level, and 
2.6% in preschool or kindergarten teaching. 

Apparently, the fellows' considerable experience and training had not been primarily 
in the subject matter areas of their respective programs, for sixty-one percent had worked as 
"specialists" in their areas for less than 3 years, while only 28% reported taking as many 
as 30 semester hours of undergraduate credit in their specialty — the presumed equivalent 
of an undergraduate major. Seventy percent had taken fewer than 10 hours of graduate 
credit in their special area, and a third reported no graduate courses at all in that area. 

Two thirds had never attended an NDEA summer institute or similar training program, and 
only a tenth had attended more than one such program. 
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It is noteworthy that only 3% of the participants had held administrative jobs 
when they entered the program. Of those not in administrative positions, 55% said they 
"probably" or "definitely" would not go into full-time educational administration, while 
only 6% said they definitely expected to do so. Upon completion of the program, these 
fellows will doubtless be in a favored position on the promotional ladder of their home 
school systems; their apparent reluctance to move into administrative positions bespeaks a 
strong commitment to classroom teaching. It will be important to examine, in future years, 
the actual career patterns of these fellows. 

2. Characteristics of Fellows* Home Communities and Schools 

Fellows were distributed according to the size of the communities they came from 
in numbers roughly proportional to the distribution in the population as a whole. Thirty- 
three percent were from communities with less than 2,500 residents, 40 ^ from 
^ cities with populations between 2,500 and 100,000, 16% from cities between 100,000 
and 500,000 population, and 19% from cities of over 500,000. Only 14% of the fellows 
identified their school system as being in a suburb or satellite city. 

At least some participants came from each part of the country. The Westerri and 
Midwestern states were somewhat over-represented, with 24% and 33%, respectively, 
of all participants; 23% of the fellows were from the North Atlantic states, about the 
same proportion as in the population at large; the Southeastern, South Central, and 
Southwestern states were somewhat under-represented, comprising only 19% of the total . 

As to school enrollment, there were fellows from schools with fewer than 200 students, 
others from schools with over 2,000, and still others in every category intermediate between 

these extremes. 

Most commonly, fellows reported that their students came from families of middle 
income. Families with low but steady income were reported next most frequently, and either 
wealthy families or those in poverty were reportedly a small minority of the clientele of most 
fellows' schools. Sixty-eight percent reported that the pupils in their home schools were 
"all or mostly white"; the remainder reported that their pupils were predominaritly Spanish- 
speaking, Indian, Negro, or a combination of two or more ethnic groups. Only about 15/o of 
the school-age children in America are nonwhite; therefore, it appears that the proportion 
of teachers in the ExTFP who came from classrooms with substantial numbers of nonwhite 
children was somewhat greater than in the nation as a whole. Since five of the 50 programs 
were for teachers of the disadvantaged, such an outcome is not surprising. 

: 

3. Ability and Interests of Fellows 

No information is available concerning the fellows' performance on standardized 
tests of ability. However, there were items on each of the questionnaires which requested 
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fellows, faculty, and directors to estimate the fellows' ability. By all three sets of judgments, 
the fellows came off extremely well. Thus, 85% of the fellows said that the participants' 
ability was above average and 95% reported that fellows seemed genuinely interested in 
the subject matter. 

The faculty and directors were asked to compare the ability of the ExTFP fellows 
with that of their institution's regular graduate students. The results of these comparisons 
are summarized in Table 1 . It is clear that the fellows were viewed very favorably by both 
sources, with program directors being consistently more favorable than the faculty. Note that 
more than half the faculty and directors reported that fellows were more industrious, more 
serious, and had greater initiative than their regular graduate students; nearly as much 
preference was given to fellows over graduate students in their commitment to the discipline 
and their ability to communicate. The faculty thought that the two groups were about 
equal in intellectual ability, while directors favored the fellows; similarly, the faculty 
rated graduate students somewhat higher than fellows in knowledge of the discipline, while 
directors' ratings were the reverse. Considering that graduate students constitute a very 
select group for comparison, these results provide an extremely favorable picture of the 
fellows' capacities. 



Table 1 . Comparisons of Fellows with Typical Graduate Students Made by Faculty and 
Program Directors 



Variable 


Source of 


Percent' Who Rated Participle 


ints 




Rating 


Better 


Equal 


Worse 


Intellectual 


Faculty 


24 


46 


28 


ability 


Directors 


29 


56 


13 


industriousness 


Faculty 


60 


33 


2 




Directors 


65 


31 


0 


Seriousness 


Faculty 


66 


27 


3 




Directors 


67 


29 


0 


Commitment 


Faculty 


46 


34 


14 


to discipline 


Directors 


67 


20 


9 


Knowledge 


Faculty 


25 


38 


30 


of discipline 


Directors 


34 


38 


22 


Ability to 


Faculty 


40 


45 


13 


communicate 


Directors 


43 


44 


9 


Initiative 


Faculty 


53 


40 


4 




Directors 


64 


27 


4 



^ Since non- respondents are not included in this table, the percentages in each row do 
not total 100. 
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It is of interest, as well, to examine what the fellows considered to be their greatest 
problem before enrolling in the ExTFP. They reported as follows: 

23% Using effective teaching methods 

20% Knowledge of the subject matter 

16% Motivating students 

T4% Determining what is most important to teach 

l2% Handling students of low ability 
~ 6 ^ Knowledge of appropriate materials 
3% Encouraging and stimulating gifted students 



One derives from these data a picture of a young, energetic, serious, industrious 
group of teachers, with considerable experience and a strong commitment to their work. 

The communities that the fellows were drawn from seem to be approximately representative 
of the nation as a whole except, perhaps, that the South was somewhat under-represented 
and that schools with substantial numbers of nonwhite students were somewhat over- represented . 

The fact that fellows were relatively untrained in the specialized subject matter of their 

programs, combined with their intellectual ability and their sincerity of purpose, suggests ^ 

that they were especially likely to benefit from their graduate work. j 

B. Characteristics of the Faculty | 

The educational and professional background of faculty members in the ExTFP ^ 

was impressive. Seventy-three percent held either the Ph.D. or the Ed.D. degree; j 

80% had taught at the college level for three or more years, 60% for six or more years, and j 

25% for more than 16 years. In addition, 37% had taught for at least a year in elementary 
school and 51% had a year or more of experience at the secondary level . Thus, many of 
the faculty were acquainted at first hand with the educational settings from which the ^ 

participants came and for which they were being trained. 

Two sets of questions bear on the quality of instruction at the different institutions, \ 

one set from the evaluation teams, the other from the fellows. When asked to comment on ) 

the qualifications of the teaching staff, 28 of the 31 evaluation tearns rated them ori the j 

"qualified" side of the continuum, two placed their ratings at the midpoint, and only one > 

team rated the staff as slightly unqualified. 

The fellows' ratings of the faculty are similar to those that would be given by 
college students in a course that was somewhat better than average. Thus, over two- 
thirds of the fellows rated the quality of lectures as good or excellent; 56% gave the same 
evaluation to seminars and structured discussions. The great majority of fellows said that 
the lectures were seldom or never over their heads (a response which might, in fact, be , 

either positive or negative), that the instructors did not talk down to them, and that the 
lectures dealt with various approaches to the subject. However, 41% believed that lectures ' 

were sometimes or usually dominated by detail or unrelated facts, and a slight majority 
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(51%) reported that the faculty had little or no knowledge of the practical problems of 
school teachers . It should be noted that this last opinion does not wholly square with the 
faculty members' reports of their own biokground in elementary and secondary school 

teaching. 

In sum, the instructors at the various institutions seem to have been quite well 
qualified. Reactions of fellows to the teaching were mixed, although on the positive 
side. The response to lectures and seminars was quite positive; on the other hand, there 
was some feeling - - based, perhaps, on the academic nature of most programs and on the 
fellows' concern about the material's applicability in the classroom — that not enough 
attention was given to the "practical" problems of teachers. 

C. Characteristics of the Programs 

The guidelines for proposals for the ExTFP emphasized en bloc programming, 
cooperation between education and subject-matter departments, and attention to the special 
character and experience of the participants; these requirements demanded of program planners 
a type of co-ordination that may not have been in effect at many institutions. Although 
the guidelines specifically stated that educational innovation was not a requirement 
for proposals, in fact the proposals which were funded were novel and extremely diveree. 

It is this diversity which is most characteristic of the group of programs as a whole • « 

50 programs, the largest number whose titles were approximately similar is four, and there 
seem to be 24 different kinds of programs indicated by the titles alone . Actwi y, except 
for the structural uniformities called for by the guidelines, there was little similarity between 
any two programs on more than a few dimensions. The dimensions along which programs 
varied may conveniently be divided into two classes, (1) organizational and situational 
characteristics and (2) goals and rationale. 

Organizational and sit uational characterist ics . The vast majority (84%) of the 
programs were^h^^tid, geographical!^, in one of ^ee areas, the Midwest (38%), 

West (25%) or Northeast (21%). The remaining 16% of the programs were scattered across 

the Southeast, South Central, and Southwest regions. 

The programs ranged in size from 5 to 25 participants, with 25 the most common 
number (34%), followed by 20 (28%) and 15 (20%). Only two programs had a participant 
group of 5 or fewer. The guidelines specified there should be cooperation between educatran 
and subject matter departments, but in every case one department bore primary responsibility 
as "home" for the program. The programs were approximately equally divided m their 
locations, with 29 programs based in education departments and 21 based in other departments 



^One reason for the novelt> and diversity among the 50 programs that were actually 
funded may be that the advisory panels used innovation and diversity as criteria in deciding 
which proposals to recommend for approval or disapproval, even though these criteria 
were not specifically set forth in the guidelines. 




- 13 - 



Great latitude was permitted to irrstitutions in the actual functioning of the progranrs. 
For instance, many awarded an MA degree at the completion of the program, others 
provided the possibility of an MA upon completion of some further work, and some mad 
no provision for on odvonced degree ot oil . Among those ovrarding on MA 'JT 

required o thesis, most did not. The choices of teaching techniques were related to h g 
of the programs but ogoin there were greot voriotions omong programs with opporently 
JJilor Cr Some programs, porticulorly those whose purpose seemed to be to upgrade 
ond eduoote ieochers in existing oreos such os History or Mothenwtics, provided o 
conventionol courses from which porticiponts selected, much os in ? 

Others particularly those programs which reportedly they were training for o new k nd 
S such o^^edio con^ or teocher of the disodvontoged, provided o reol 

“bloc" of coupes identical for each porticipont, and often very different from 
tourht elsewhere in the institution. Seminors, workshops ond proctico were in general more 
common in the lotter programs thon in conventionol groduote sequences. 

As to the gools which programs pursued, their diversity hos olreody been mentioned . 
Proorams ranged from fairly conventionol, though certainly important, attempts to upgrad 
fh?3e™t knowledge, ond techniques of teochers of English, History, or Geography 
to th~ c'e^tion 7o % person "n the educotionol setting" such os o centralized mecho 

peciolTsror on educotio'nol s'^tems onolyst. There were fWe programs to tram teoche. o 
different disodvontoged groups, including rural Aloskon Indions, Texos Mexicon-Americons, 

and Harlem Negroes. 

In 18 pragrams the principol emphosis wos on secondory school 
emphasis was at the primary level, and at least four covered both levels. In addition, ther 
wera^ 10 programs for the training of coordinating or odvisory personnel, such os guidonce 
counselors, media specialists and school librarians. 
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IV. A Profile of Reactions to the Program 



This section of the report will consider reactions to the Program by fellows, 
faculty members, directors, and evaluators. It begins with the degree of satisfaction that 
was expressed, then proceeds to various judgments of the Program's effectiveness. A 
discussion then follows of how and whether the various programs implemented the three 
requirements regarding program structure that were spelled out in the guidelines: the 
importance of an en bloc approach, the necessity fpr subject-matter and teacher education 
deportments to work closely with each other, and the requirement that relationships be 
established with the local school systems. Finally, we will discuss what might be called 
the "strategy of operation" adopted by the different programs, including the extent to 
which the programs made use of the extensive experience of the fellows, the amount of 
competitiveness that was fostered among fellows, their work load, the extent of faculty 
involvement in the program, and the amount of innovation in the curriculum. 



A. Satisfaction and Effectiveness 

It is not easy to differentiate between a person's satisfaction with an educational 
program and his judgments of its effectiveness. Presumably, the two kinds of responses 
should vary with one another — indeed, we shall see in the next section that they co- 
vary to a remarkable extent; nevertheless, the distinction seems worthwhile, for satisfaction 
with a program refers to one's overall emotional response; a program is judged to be effective 
or ineffective according to whether it achieves the goals the respondent expects of it. 

Thus, a program might conceivably be effective without necessarily producing high levels of 
satisfaction among the participants, and vice versa . We consider, first, the extent of 
satisfaction with the ExTFP, then judgmet>ts of its effectiveness. 

1 . Satisfacti o n With the Pro gram 

Two questions which appear to reflect satisfaction with the ExTFP were included 
in substantially the same form in the questionnaires given to the fellows, to the faculty, 
and to the directors. One of these deals directly with reactions to the Program: 

Which of the following alternatives best describes your reaction to the 

Experienced Teacher Fellowship Program? 

It was a stimulating and interesting experience throughout. 

It was usually stimulating and interesting . 

It was only occasionally stimulating and interesting. 

It was seldom or never stimulating and interesting. 

A comparison of the responses of the three groups is given in Table 2. Clearly, 
the reaction in every group was overwhelmingly favorable, with faculty members somewhat 
more favorable than fellows, and directors the most favorable of all . It should be under- 
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lined that among even the least enthusiastic group, the fellows in the program, 82% reported 
that the ExTFP was either usually stimulating and Interesting or stimulating and 
interesting throughout. 



Table 2. Ratings by Faculty, Directors, and Fellows of How Stimulating and Interesting 
They Found the Program 



Source 


Percent Saying ExTFP Was Mimulating and Interesting 


Throughout 


Usually 


Occasionally 


Seldom or Never 


Faculty 


42 


46 


8 


1 


Directors 


60 


38 


2 


0 


Fellows 


32 


50 

,1 — — 


16 


1 

1 



The second item that may be considered a measure of satisfaction asked faculty 
and fellows for a comparison of the Program with a typical eight-week summer institute. 
Faculty members and directors who had faught in such programs, but not those who hod not, 
were asked to compare the ExTFP with NDEA or NSF academic year or summer insti utes. 
Since fellows hod earlier reported whether they hod ever attended such institutes it was 
possible to divide them into two groups: those with and those without prior institute 

experience. The four sets of comparisons of the ExTFP with other institutes are presented 

in Table 3. 

Again, it is evident that the response to the ExTFP was overwhelmingly favorable. 
On this item, the fellows were more likely than the other two groups to view the ExTFP 
as superior to other institutes, and those who had not taken part in such institutes were 
somev.'hat more favorable than those who had. Again, the directors showed somewhat 
greater approval of the ExTFP than did the faculty. Of greater importance than these 
inter-group comparisons, however, is the fact that only a tiny fraction of fhe respondents 
in any group felt that other kinds of institutes were superior to the ExTFP, 



> It must be emphasized that these judgments are almost cerlainly expressions of 
satisfaction with the ExTFP instead of a true reflection of the relative effectiveness °f 
Program gnd other institutes. Many factors were involved in these responses: fellows m the 
ExTFP had committed a full year or longer to that program; their stipends were larger than 
those paid in the summer institutes and they extended over a full year instead of eight weeks, 
many the ExTFP fellows were receiving advanced degrees or credit toward such aegree'. 

In short ExTFP fellows and staff were comparing a present valued experience o a is an on , 
their comparisons can hardly be considered unbiased judgments of effectiveness; as expressions 

of satisfaction, however, the results are impressive. 
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Table 3. Comparisons by Faculty, Directors, and Fellows of ExTFP and Other Institutes 





Percent of Group in Each Response Category 


Source 


ExTFP 

Superior 


ExTFP 

Somewhat 

Better 


About the 
Same 


Institute 

Somewhat 

Better 


Institute 

Superior 


Faculty with Institute 
Experience (N=57) 


28% 


21% 


37% 


9% 


5% 


Directors with Institute 

Experience (N=21) 

• 


52% 


19% 


19% 


5% 


5% 


Fellows without Institute 
Experience (N=486)^ 


80% 


13% 


5% 


1% 


1% 


Fellows with Institute 
Experience (N-312) 


65% 


18% 


8% 


6% 


} 

CM 



number of respondents omitted this item; the totals on which the responses 
ore based are those who actually responded. 



2 . Judgments of Program Effectiveness 

Unfortunately, the questionnaire method does not provide a fully satisfactory means 
of determining the effectiveness of an educational program. Subjective ratings can 
provide, at best, imperfect estimates of what a student has learned from a set of material 
or of how well his new knowledge will be applied when he returns to his earlier role. 
Whether a student has profited a little or a great deal from a program should be assessed 
by comparing what he knows at the program's end with what he knew at its beginning; 
similarly, whether he will apply what he has learned can be determined adequately 
only by observing his performance on his home grounds. Nevertheless, in the absence of 
more reliable measures of program effectiveness, the subjective ratings that are 
obtained in questionnaires are considerably better than no estimates of effectiveness at all . 
Especially when the respondents are experienced judges of the effectiveness of educationa 
programs — and such is certainly the case in the present study - one can expect their 
replies to the questionnaire to relate positively, if imperfectly, to more objective measures 
of program effectiveness. With the material at hand, we have no choice but to use 
questionnaire ratings of effectiveness. We cannot estimate the degree to which these 
ratings correspond to the "true" effectiveness of the different programs; nevertheless, 
we can reasonably assume that there is considerable validity in these judgments. 
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There vwis only one question relating to effectiveness which asked for approximately 
the some kind of judgment from faculty, directors, fellows, and evaluators The 
question took var ed considerably from one questionnaire to another. In the 
dbeclor questionnaires, respondents were asked: "In your opinion were the emotional 
needs of *e porticipants met by the program?" Fellows were first asked to check, 
from among seven different teaching problems, the one that had concerned them m s 

before the%nrolled in the ExTFP. In the next item they were asked: To what 

extent did^the program this year meet that problem?" Finally, evaluation teams w r 
askL to rote, onl seven-pLt scale, whether the program seemed to -eeMhe needs 
of the participants. Responses of the four groups to these items are presented in Table 4. 

Table 4. Estimates by Faci-i Directors, Fellows, and Evaluators of Whether the 
ExTFP Met Fello - Needs 




Fellows 



Evaluators 



Were the educa tional needs of participants met? 
Probably 



To what extent did tlie program meet (your major teaching) 
problem? 



To a great 
degree 



35% 



To a moderate 
degree 



39% 



To a slight 
degree 



21 % 



6 % 



Did the orog rom seem to meet the rieed s of participants 
uCTI 1 Neutral 



0 



51% 



32% 



16% 



It is evident that the directors and faculty were both confident that the fellows 
educational needs had been met, with the directors, once more somewhat P“'f^e 
than the faculty. The confidence of these two groups in the effectiveness of th g 
is further reveoL by their responses to two other questions ^hen oskedjhejh^^^^^^ ^ 
the ExTFP resulted in the participants becoming better teachers, 7 ^ , l r n 

vs-' onS " 4 % werl unce^oin (presumably for lock of direct obser^j-- 
teaching); the corresponding proportions for directors were “"d 13^. Sim y, 
92% of the faculty and 100 % of the directors reported, in another item, that the overall 

program was either valuable or very valuable for the participants. 
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It may be seen in Table 4 that 74% of the fellows reported that the 
had met their^maior problem to at least a moderate degree. While this is a substanti 
majority, it also eaves one fellow in four feeling that his major 

“I totly 0 slight degree (21%) or not at oil (6%) This outcome should not be token 
as evidence that one fourth of the fellows thought the program was ineffective. 

or extremely useful; 18% said it hod been somewhat useful, and only 3/o called the program 
not useful at all. 

From Table 4 it appears that evaluators were somewhat le» ff/ 

effectiveness of the programs than were the other three groups. Just oyer half of he 
elluotion teams said the institutions they visited had met 
five teams said these needs were less than adequately met. These last five 
e^lurr! remarked on the similarity of the programs they visited to ^®9^;; 
and aroduate training, and also on the lack of adequate practicum experience ,T !u " 
more favorable view of the programs was expressed in evaluators' judgments ° * ® _®'' 
the fellows would be able to apply what they had learned when they returned to the.r 
scLol Twenty-three of the 31 teams reported in the affirmative, four placed their 
r-n I at the mfdaoint and only four teams said that the students they observed were 
^nlewhat unUrify to be oWe Jopply what they had learned. Several teams remarked 
that their judgments were less favorable than they might have been because they f^re 
h^t radit'orwiist or money-conscious school systems might resist the introduction of 

f fhp material the fellows would bring bock with them from their year of training. 
mL indie™ on^^^ of I effectiveness of the Program comes from on 

^Xs s S he general comments they wrote at the end of the evaluators form J se 
comments rev Jed a clear acceptance, by all evaluators of the general va ue of the 
Ex7p Even those few evaluation teams which expressed rather extreme criticism of 
Jhe iLnSns C felt that the fellows had profited in some d®9r®® fr®-" !h®^ 

year of study; their criticisms frequently stemmed from the conviction that^stan i y 
Lore could have been accomplished had the program been conducted differently. 

In summary, it is clear that there was general satisfaction with the Program, and 
widespread agreeLnt that it was an effective educational venture. It should not be 

.K., ,h. f 

Z vTer"hatproi J favorably. Despite this built-in bias, the overwhelming favorableness 
of the ooiniom given by these three sources strongly supports the conclusion that the 
MFP ^s T^Mying Lperience for fellows and staff alike . The evaluation teoi« 

had no personal involvement in the outcome of their evaluation. J 

assianed a critical role, and they measured the programs against high standards of • 

The?r generally positive evaluations provide further evidence of the program s effectiven . 
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B. Structural Aspects of the Programs 



Doubtless because of the lack of specificity in the guidelines, there was considerable 
variation in the pattern of organization adopted by different institutions in implementing 
the ExTFP. Since only a limited number of items dealt with such questions, many of 
these differences in program structure went unrecorded. Only three structural aspects 
of the programs will be dealt with here, all of them specifically discussed in the guidelines 
for proposals: the utilization of the en bloc approach, the relationship among teacher- 
education and content departments, and the pattern of relationship with local school systems. 

1 . The en bloc Approach, Group Solidarity, and Morale 

Only one question dealt specifically with whether the en bloc approach was 
adopted by the various institutions; this was an item in the Visitors Evaluation Form which 
said "Unlike conventional graduate programs, the Experienced Teacher Fellowship Program 
is based upon a block or group program approach. The intent is to use the group to 
enhance learning by building morale and esprit de corps . Has this been successful?" 

Evaluators' responses make it clear that the en bloc approach was, indeed, 
successful: 12 evaluation teams reported that the approach was extremely successful and 
13 others rated the approach as successful, but not extremely so. Only three teams said 
that the approach was in some degree unsuccessful . 

Although the en bloc approach was not mentioned in the other three questionnaires, 
all three groups were asked whether there was a feeling of group solidarity among participants 
in the program; in addition, respondents were asked to rate the overall morale of the 
participants. The responses of the three groups are presented in Table 5. 



Table 5. Ratings by Faculty, Directors, and Fellows of Group Solidarity and Morale 





a . Was there a feeling of group solidarity? 
Percent Answering 


Source 


Strong 


Considerable 


Some 


None 


Faculty 


53 


38 


5 


1 


Directors 


67 


27 


7 


0 


Fellows 


46 


38 


14 


1 




b. How would you rate fellows' morale? 




Source 




Pretty 

High^ 


Average 


Low and 
Very Low 


Faculty 


23 


51 


14 


4 


Directors 


38 


42 


18 


2 


Fellows 


22 


38 


27 


10 
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By ©very measure, solidarity and morale were high. Again, the directors 
estimates of both variables were somewhat higher than those by the faculty, and the 
faculty's estimates were higher than those by the fellows; but even among the fellows, only 1% 
said there was no feeling of solidarity and only 10% reported that the group's morale 
was below average. It should be mentioned that there was considerable homogeneity 
within groups of fellows in their judgments of solidarity. That is, reports of relatively 
low solidarity were concentrated in particular institutions; they were not made by 
social isolates scattered among a number of programs but probably reflected, instead, 
a real lack of solidarity in a few of the programs. 

2 . Cooperation between Departments 

It will be recalled that the guidelines for the ExTFP specifically called for cooperation 
between subject-matter and teacher-education departments in conducting the program. 

Three questions, one each from the fellows, the directors, and the evaluators questionnaires, 
asked whether such cooperation was achieved. 

In some institutions, such cooperation apparently did not extend much beyond 
consultation on the initial application. When asked whether the director of teacher- 
education was involved in*ie operation of the ExTFP, only 26% of the program directors 
replied that he was either quite involved or very much involved; just over half said he 
was not very involved, and another 20% reported that he was not involved at all . 

This question, of course, asked only about the director of teacher education, not whether 
there was cooperation with others in lesser positions in the teacher— education hierarchy. 

In fact, when they were asked to describe the cooperation they received from other 
academic departments, 91% of the directors said it was either quite good or unusually 
good. This suggests that some collaboration must have taken place between teacher- 
education and subject-matter departments, or at least that the directors thought so. 

Evaluators' reports indicate that interdepartmental cooperation varied widely from one 
institution to another. While only one evaluation team reported very close cooperation 
between the two departments, another 12 placed their ratings at the cooperative side of 
the continuum. On the other hand, 15 of the evaluators' judgments were on the uncooperative 
side; in five of these institutions evaluators said there was no cooperation at all between 
teacher-education and subject-matter departments. 

Responses by fellows show much the same picture as those by evaluators. Eighty-four 
percent of the fellows reported that their instruction involved more than one academic 
department. Exactly half of these, 42% of the total group, said the material was coordinated 
either quite well or extremely well; the other half, again 42% of the total group, said the 
material was either not coordinated too well or was not coordinated at all . We should 
recognize, however, that this question does not bear directly on the point at issue, for the 
second department which the fellows had in mind need not have been the department of 
teacher education. One other item on the participants' questionnaire had at least a 
tangential bearing on this question. In response to a question about the relative emphasis 
on subject matter and teaching methods, the majority of fellows, 64% said that the balance 
was about right, 28% reported that there was too much emphasis on subject matter, and 
only 3% reported too much emphasis on teaching methods. 
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3. Cooperation with Local School Systems 



Only one question dealt with this topic: evaluators were asked to report how 
extensive the relationships were between colleges and universities and cooperating school 
districts. Again, there were great differences from one institution to another: fourteen 
institutions were rated on the low end of this continuum, 12 at the high end, and five at 
the midpoint. In the evaluators' written comments about the programs, some of the most 
caustic had to do with the lack of practicum experiences available to fellows in those 
institutions without relationships to cooperative school systems. 

To summarize, the en bloc approach seems to have been effective in most of the 
: institutions that participate*d in the ExTFP; it was accompanied by a high degree of 

solidarity in most of the groups, and by reports of high morale among the fellows. In the 
matter of cooperation between teacher-education and subject-matter departments, the 
program does not come off so well: in some institutions there was substantial cooperation 
of this sort, in others there was little or none. The same wide range held true for the 
I extent of cooperation between the participating institutions and local school systems. 

C. Operating Strategies 

Even when programs have the same formal structure, there may be extensive differences 
in their mode of operation along a variety of dimensions. For example, graduate programs 
in American universities are known to differ in the degree of competitiveness that they 
foster among graduate students; it would be expected that ExTFP programs would also vary 
in this regard. Similarly, universities, and by extension the ExTFP programs, differ widely 
in the extent of faculty involvement with students and in faculty commitment to instruction. 

Beyond this, the nature of the ExTFP suggests that there are other dimensions 
along which variation may be expected. The experience of teaching for a number of 
years has provided each fellow in the Program with a degree of specialization in his field, 
an awareness of the problems that are involved in teaching his subject, and an intellectual 
and emotional maturity that set him apart from the typical graduate and undergraduate 
student. His response to the program of courses that is offered should depend, in considerable 
part, on whether and how that program builds upon and utilizes his extensive experience. 

We turn now to a discussion of differences among institutions in these aspects. 

1 . Utilization of Fellows' Background 

All four sources — fellows, faculty, directors, and evaluators — were asked, in 
one way or another, whether they felt the program had taken advantage of the rich experience 
and prior preparation of the fellows. Although these questions were phrased differently 
; for different populations, the four sets of responses have been grouped so that they are 
roughly comparable; they are presented in Table 6. 




- 22 - 






Table 6. Judgments of Whether Programs Built Upon Fellows' Experience and Preparation^ 



Source 


Percent of Each Group Responding 


Yes 


Uncertain 


No 


Faculty 


66 


23 


6 


Directors 


89 


7 


4 


Fellows 


63 


24 


12 


Evaluators 


32 


16 


52 



^ In the faculty and directors questionnaires, the alternatives for respondents 
to check were "Yes, " "Uncertain, " and "No." Fellows who said that the program 
usually or consistently built on their backgrounds have been scored as replying "Yes, " 
those who said it rarely did so are scored as "Uncertain, " and those who said it was 
unconcerned for their background are scored as saying "No." Evaluators' responses 
are recorded as "Yes, " "Uncertain," or "No" according to whether their judgments 
were on the positive side of the midpoint^ at the midpoint, or on the negative side. 



Clearly there was a considerable difference among sources in their judgments 
of whether the programs took account of fellows' backgrounds. Program directors were 
most likely to say that the programs had built on fellows' backgrounds; faculty members 
and fellows were somewhat less certain, but the clear majority of these two groups agreed 
with the directors that the program utilized the fellows' prior experience. Real disagreement 
with these judgments was shown by the evaluation teams, over half of whom said that 
the institutions they visited had not designed their programs to take account of fellows' 
experience. The comments of those teams that were critical, on these grounds, of the institutions 
they visited were examined in some detail. In an interim report, based on about half of these 
responses, comments by critical evaluation teams were summarized in a manner which 
holds true after the remainder of the data have been collected: 

Most commonly, [^valuators who were critical] remarked that fellows 
were treated like regular graduate students, complete with the institution 
of multiple-choice examinations and competition for letter grades, 
with the prescription of a fixed schedule of courses, with little tailoring of 
individual programs to the needs of individual fellows, and with little 
or no opportunity for fellows to exchange ideas with one another about 
their own experiences. In short, . . .these institutions offered substantially 
the same kinds of programs they had always offered. 
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Why the other three sources at those same institutions should so strongly disagree 
with the evaluators' judgments remains unclear. Perhaps in their involvement with one 
institution, they were unable to conceive of the range of alternative policies that the 
evaluators envisioned; as a consequence, policies that these sources thought were major 
concessions to the fellows' experience may have been viewed by evaluators as modest 
efforts, at best. That faculty members and program directors actually did believe they 
had utilized the fellows' experience is made evident by their replies to a question asking 
whether their program was modified to take advantage of the experience and background 
of participants. Sixty-two percent of the directors and 36% of the faculty answered 
this question affirmatively, 9% and 40% were uncertain, and only 27% and 16% answered 



'no. 



M 



2. Competitiveness and Work Load 



Our interest in discussing the extent of competitiveness and the size of fellows' 
work load is not to determine whether participants in the ExTFP worked or loafed, but 
to see whether they thought they were overworked and how intense was the competition among 
fellows. Unfortunately, the question that was asked of faculty members and directors 
appears to have been relevant to the first question, not to the second. It asked whether 
the students worked hard during the year. Eighty-four percent of the program directors 
and 79% of the faculty replied "yes, " a bare 13% and 15% respectively of the two groups 
said "yes, too hard, " and the remaining few respondents were uncertain or reported that 
fellows had not worked hard. In retrospect, these replies seem to reflect the respondents' 
approval of fellows in their programs more than their evaluation of the fellows' work load. 

By contrast, fellows were about evenly divided between the opinion that their 
work load was about right (52% of the respondents) and the view that it was too heavy to 
allow completion of assignments and independent work (47%). Ratings by the evaluator 
teams also indicated that the work load in some schools was heavy: 10 of the 31 teams 
reported that the work load at the institution they visited was about right, the remaining 21 
said that the work load was in some degree too heavy. As to competitiveness, 60% of 
the fellows said that the level of competition in their program was either quite high or 
extremely high, 31% said it was about right, and only 8% said it was either low or very low. 
It should be mentioned that there was considerable homogeneity of judgments on these 
items among fellows in the same programs; that is, in certain programs almost all of the 
fellows said the work load was too heavy, in others, almost all said it was about right. 

A cogent comment on these judgments is the remark that graduate education 
involves a great deal of work wherever it occurs. Indeed, many of the evaluation teams 
who rated the work load as somewhat too heavy observed that such is the norm in graduate 
school, and that after the program was over fellows might cherish their experience the 
more for the fact that strenuc is demands had been made of them. Nevertheless, it 
appears that some institutions did require far more work than their fellows could produce, 
thereby introducing severe emotional stress into the academic program. 
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3. Involvement of the Faculy 



Assignment to teach in the ExTFP might have been accepted by a faculty member 
as simply another unit in his teaching load, requiring no change in the kind of material 
he presented/ in the way he presented it, or in his involvement with the students. 
Alternatively, it might have been viewed as a special challenge which called for a 
somewhat different orientation toward both the subject matter and the students. The 
tone of the guidelines makes it clear that their intent is for the second attitude to be 
dominant among the staff of the ExTFP. There were no items in any questionnaire that dealt 
directly with the mode of orientation of the faculty, but there were a number that skirted it; 
we turn our attention now to those items. 

Faculty members and program directors were asked whether they found the ExTFP 
a challenging and satisfying experience. As may be seen in Table 7, the majority of both 
groups replied in the affirmative to both questions. Program directors, once again, were 
somewhat more enthusiastic than the faculty. 



Table 7. Ratings by Faculty and Directors of Whether the Program Was Challenging 
and Satisfying 



Source 


a. How Challenging was the ExTFP? 


Extremely 


Somewhat 


Not Very 


Not at All 


Faculty 


47 


42 


5 


0 


Directors 


64 


31 


0 


0 


Source 


b. How satisfying was the ExTFP? 


Extremely 


Somewhat 


Not Very 


Not at All 


Faculty 


57 


40 


3 


1 


Directors 


71 


24 


0 


0 



The enthusiasm of the directors and faculty for teaching in the program was 
clearly picked up by the evaluation teams, for 22 of the 31 reported that the director 
and staff were challenged and stimulated by the program; only 8 gave judgments that 
fell toward the opposite pole of the continuum. 

Fellows were not asked about whether the staff was challenged by the program, but 
whether they were accessible and helpful to students. Their responses were overwhelmingly 
favorable on both counts: 94% reported that the staff was either usually or always accessible, 
95% said it was either usually or always helpful . 



Although, as we have seen, the faculty and directors reported being challenged and 
stimulated by the Program, they were less likely to report that the experience affected 
their own professional and intellectual growth. As may be seen in Table 8, only 41% of the 
faculty said their professional development was furthered either "greatly" or "very greatly" 
by the experience; only about 30% said it added greatly or very greatly to their intellectual 
growth and to their skill as teachers. Again, directors were more generous in their 
estimates of how much benefit they derived from the Program; 51% said it added greatly 
or very greatly to their professional growth, but only about a third judged it had a 
comparable effect on their intellectual growth or their skills as teachers. In each ca^ie, 
respondents' judgments of the benefits they derived from the program were less favorable 
than their ratings of the challenge and satisfaction they felt. It is not clear how much 
weight should be given to these results. Perhaps the experienced University teacher does not 
ordinarily profit in these ways from his teaching experience. In any case, it is clear that 
the experience may have been stimulating and challenging but was not viewed as educational 
for the majority of the staff. 



Table 8. Judgments by Faculty and Directors of the Program's Effects on Their Own 
Development 









Percent Resppnding^ 




Item 


Source 


Very 

Greatly 


Greatly 


Moder- 

ately 


Little 


IVery 

Little 


28. Add to profession 


Faculty 


14 


27 


35 


14 


5 


growth and develop- 
ment? 


Director 


18 


36 


42 


2 


0 


29. Add to your intellectua 


Faculty 


11 


19 


40 


18 


7 


growth ? 


Director 


9 


29 


56 1 


4 


0 


30. Add to your skill as 


Faculty 


10 


19 


42 


19 


4 


a teacher? 


Director 


9 


24 


53 


4 


4 



^Percentages in each row do not total to 100 because non- respondents are not 
included. 



4. Departmental Innovativeness 



It was not required that institutions prepare thoroughly innovative proposals in 
order that their programs be funded under the ExTFP. Instead, substantial ly traditional 
proposals were examined in competition with completely innovative ones; approyal or 
disapproval for funding was not determined in terms of the program's rjovelty, but in terms 
of how effective it seemed likely to be in furthering the education of experienced teachers. 

In the viewof program directors, there was innovation in the great majority of 
the institutions. Of 42 directors who responded to this item, 32 said they had seen 
imaginative teaching methods and practices in their programs, 6 were unsure, and only 4 
reported that they had not seen such practices. As usual, judgments by faculty members 
were less extreme: of 174 who responded to the item, 79 reported innovations, 22 were 
unsure, and 73 reported none. 

The evaluation teams agreed more with the faculty than with the directors: 

12 teams rated the institutions they visited as being on the innovative side of the midpoint, 

12 ratings were on the noninnovative side, and the remaining 7 were exactly 'it the 
midpoint. None of the evaluator ratings fell in the most extreme categories, those which 
indicated either a great deal of imagination and innovaiion or none at all . 

In summary, there was disagreement among sources in the extent to which they 
thought the programs had utilized the background and experience of their fellows; the 
least enthusiastic source of ratings, the evaluators, judged that there were more 
Institutions which did not make sufficient use of the fellows' experience than there were 
which did. Apparently, there were systematic differences between institutions in the 
amount of work they assigned their students: every institution required a considerable 
amount of work, but some assigned an enormous amount. By all accounts, the faculty 
was challenged and stimulated by the program, accessible and helpful to the fellows. Finally, 
it appears that programs were neither thoroughly innovational nor stodgily traditional, 

5. Effects on Institutional Development 

One cf the benefits the Program might have wrought, indeed, >ne of the effects 
that was envisioned initially, was a strengthening of the participating institutions themselves, 
particularly in their on-going teacher education programs. Obviously, changes in the 
pattern of teacher education will have effects upon the preparation and later performance of 
those who are trained; therefore, it is important to determine whether the Program actually 
influenced educational patterns in the host institutions. Important though this question may 
be, it is uncertain whether it can be answered adequately by the present study. These 
data were collected in the first year of the Program's operation, barely eight months 
after it was instituted . Whatever effects it may ultimately have upon procedures for 
teacher training, these effects are not likely to have taken place by the time these data 
were collected. Consequently, the conclusions we may draw about such effects must 
inevitably be tentative. 
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Of the five items that dealt with this issue, two were global judgments, by the 
directors and the faculty, of the Program's "value to the institution." One evaluator 
item asked about the Program's contribution to the plans of the Department, and two 
items, from the evaluators and the directors, asked specifically about the Program s 
"impact on the on-going teacher education program." Thus there were three distinct, if 
related, issues involved in these items: value to the institution, contribution to department 
development, and impact on teacher education. 

As was so often the case, judgments on these questions varied remarkably from one 
source to another (Table 9). More than three fourths of the faculty members and the directors 
reported that the Program was either "valuable" or "very valuable" to their institution. 

On the other hand, among evaluation teams as many said the program's contribution 
to departmental development was slight or non-existent as said it was moderate or great 
(36%, in each case). The same divergence of opinion was found in judgments of the Program's 
effect on teacher education: 62% of the directors said its impact on teacher education 
was relatively large and only 23% said it was small; the corresponding figures for evaluation 
teams were 16% and 58%. 



Table 9. Judgments by Faculty, Directors, and Evaluators of the Program's Effects Upon 
Institutional Development 





a . Percent saying Program's 


value to institution' 




Source 


Great 


Moderate 


Undecided 


Slight 


None 


Directors 


42 


44 


9 


2 


0 


Faculty 


29 


47 


13 


5 


1 




b. Contribution to departmental and institutional development 


Evaluators 


26 


10 


29 


26 


10 




c . Impact on teacher education 


Directors 


24 


38 


13 


16 


7 


Evaluators 


6 


10 


26 


26 


32 



^To make results from three questionnaires comparable, responses to the Visitors 
Evaluation Form have been classified as follows: checks in either of the two most favorable 
categories are scored as judgments that the program had great effects; those in the third most 
favorable category are scored as reflecting moderate effects; those in the center category 
are classified as "undecided"; and responses are considered o> imputing slight effects or 
none according to whether they fell in the third or in the two most unfavorable categories. 
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These differences in judgments by the different sources probably reflected a number 
of factors. For one thing, the directors and the faculty had more at stake in the Program 
than did the evaluators; no doubt this involvement influenced their judgments in a favorable 
direction. It is probably true, as well, that the different sources used different criteria 
to assess the effects of the Program; what looked like a remarkable advance in the context 
of a particular institution may have seemed trivial to an outside observer. Paradoxically, 
some of the disagreement in judgments may have occurred because institutional changes 
had been made before the ExTFP was undertaken: a few evaluation teams said the Program 
had little impact on teacher education because the existing procedures were advanced and 
effective; directors at those institutions attributed more influence on teacher training to the 
Program than did the evaluators. 

One determinant of whether a change was effected in the host institution s teacher- 
training practices was the division of the institution in which the ExTFP was located. All 
five of the institutions that evaluation teams rated above the midpoint on "impact on 
teacher education" were based in education departments. Judgments by evaluation teams 
of the extent to which departmental and institutional development were affected showed the 
same patterns: 7 of the 8 programs in which evaluation teams said the effects were greatest 
were based in education departments. Not surprisingly, then, educational changes were 
more likely when Departments of Education were directly responsible for the Program; 
stated somewhat differently, educational programs that were located in Liberal Arts 
departments did not have immediate effects on the policies of education departments. 

D. General Summary of Impressions of the ExTFP 

It is clearly evident from the results that have been reported in this section 
that the sources' evaluations of the ExTFP varied directly with their involvement in the 
program. Program directors, who probably had the most at stake in the enterprise, 
were thoroughly enthusiastic, not to say Pollyannaish, in their ratings. Regular faculty 
members and fellows, who were somewhat less personally involved than the directors, made 
judgments that were a little less enthusiastic than those of the directors. Evaluation teams, 
who sppnt only two days viewing the programs and who maintained calculated objectivity 
as their ideal, were able to temper their enthusiasm with criticism. 

Yet it is the burden of this report that all four sources, including the evaluators, 
ptxjduced predominantly favorable judgments of the program. Furthermore, the responses 
of those who were involved in the program are not to be discredited simply because of their 
involvement. In all but a very few institutions, it appears, a group of highly qualified 
teachers were brought together with a group of intelligent, hard working, experienced, 
thoroughly committed students. When circumstances also promoted the development of 
strong group solidarity and high morale among the fellows, a truly impressive educational 
experience probably occurred. Even when the social context v/as less than ideal, the 
juxtaposition of a first-rate student body and a better- than-competent faculty doubtless 
produced educational effects that were considerably above the average. 
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V. Correlates of Effectiveness and Satisfaction 



We have seen that satisfaction with the ExTFP and judgments of its effectiveness 
were both very positive. Nevertheless, there were consistent differences from one program 
to another in the extent of satisfaction of the various respondents and in the ratings of 
effectiveness that they gave. Our purpose in this section is to examine the relationship 
of other variables to judgments of effectiveness and satisfaction. 

Two sources of evidence one qualitative, the other quantitative are used 
in this analysis. The qualitative material consists of comments by evaluation teams about 
the influence that program directors had upon the effectiveness of the ExTFP. This chapter 
begins with a discussion of these comments. The quantitative material is made up of 
correlations among responses to the various questionnaires; the analysis of this material 
constitutes the bulk of the chapter. A detailed summary is presented at the end of the 
chapter; readers who are unfamiliar with correlational materials may find it helpful to 
read this summary before examining the correlation tables in detail. 

A. The Role of the Director in Program Effectiveness 

Analysis of the role of the program's director in the operation of the ExTFP was 
not systematically built into the questionnaires and the evaluator'^ ratings. Nevertheless, 
reports from evaluation teams made it clear that the actions of the director were frequently 
crucial to the succesi; or lack of success of individual programs. Once this became clear, 
the written comments of the evaluation teams were examined in detail to make whatever 
inferences y^ere possible about this topic. Analysis of these comments may be summarized 
as follows: ' 

It is apparent from the reports of evaluation teams that the quality of directors 
had a major impact on the conduct of programs. In general, when the evaluators 
commented on the ability, dedication, enthusiasm, availability, and seriousness of 
directors, they also rated the programs as effective and productive. When comments 
were made about the director's lack of status in the institution, when the directorship 
changed between the time of application and the time the program began, when tension 
ruled between the director and his staff, the program was characterized as weak, poorly 
planned, poorly integrated, and unproductive. 

Because there was no provision in the guidelines stipulating that the director 
be given released time for his administrative duties, many directors lacked time to 
carry out their duties and lacked furxJs for necessary supporting work. In some cases, 
the director functioned as a coordinator rather than an administrator, with neither the 



1 



Professor William Engbretson carried out this analysis and drafted the summarizing 



statement. 
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power nor the funds to conduct the program as it had been represented in the proposal. 

In sum, programs seemed to be most effective when the director was deeply involved 
in the program's goals and was able both to devote sufficient time to administrative 
duties and to foster cooperation and respect from participants and faculty. 

The frequency and urgency of these comments suggests that special consideration 
should be given to this key role in future studies of the ExTFP and in the organization of 

individual programs. 

B. Some Comments on Correlational Meihods 

1 . The Nature of the Data 

To determine the relationships among variables, for every institution the arithmetic 
mean was computed for the judgments made by each source on the items that were of 
interest This permitted institutions to be arrayed, for example, according to the average 
degree of satisfaction that the fellows expressed, accordir^ to the average faculty rating 
of effectiveness, and so on for a substantial number of variables. Product- moment 
correlation coefficients were then computed among these variables. 

It should be obvious that two variables cannot be correlated with one another 
unless there is at least some variation in the scores on each from one observation to another. 

If all of the scores on one item fall at the identical point, then responses to that item 
cannot possibly co-vary with responses to some other item. On many of the items that ^ 
dealt with satisfaction and effectiveness the responses of directors showed next to no varia ion 
being largely concentrated at the most favorable alternatives. For this reason, directors 
responses will not be included in the correlation matrices that are presented in this section. 

There remained responses by faculty and fellows at 47 institutions, and responses 
by faculty, fellows, and evaluators at 31 institutions which were visited . It seemed 
clear that our interpretation of the results would be substantially strengthened by 
including a discussion of the correlations of evaluators' judgments with those made by 
fellows and faculty members. However, correlations based only on the 31 programs ^ 
that were visited might, because they ignored 16 other institutions, give a distorted p^icture 
of the true pattern of relationships among variables. To make sure that this v/as not tlie 
case, two correlation matrices were computed, one based on responses by faculty and 
fellows in all 47 institutions, the other based on responses by faculty, fellows, and 
evaluators in the 31 schools that were visited. A comparison of the correlations between 
identical pairs of variables in the two matrices showed that very similar results were obtained 
Therefore, only correlation coefficients based on the 31 programs that were visited will be 
used in the results that are reported below. With a set of 31 observations, a correlation 
of about .35 is required for the inference that it differs from zero by an amount greater 

than would be expected by chance. 
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2, Interpreting the Correlation Coefficients 



When two variables show a sizeable correlation, it is often tempting and sometimes 
reasonable to conclude that they are somehow causally related. This temptation should 
be indulged with caution, if at all; causal relationships cannot be established by correlational 
technic)ues. For example, we shall see that there was a high positive correlation between 
fellows' morale and their judgments of program effectiveness: in programs where morale was 
high, fellows' judgments of effectiveness was high, when morale was low so were fellows' 
judgments of effectiveness. Clearly, however, this does not mean that high morale 
produces an effective program. It is equally likely that the causal chain goes the other way, 
that morale goes down when a program becomes ineffective or up as effectiveness improves. 

It is also plausible that the two variables interact, so that some degree of ineffectiveness 
depresses morale, which makes for even less effectiveness, decreasing morale still further. 

The point is that one should be cautious in interpreting correlations. The results that will 
be reported below often seem to point toward ways by which programs can be improved; we 
believe, in fact, that they offer suggestions for improvement. But these suggestions must 
be examined intelligently, not accepted uncritically as a consequence of an impressively larg 
correlation coefficient. 

A final point must be made. It has long been known that when judgments are 
made on several variables, all of which have a desirable and an undesirable pole, a 
built-in correlation is introduced. A respondent who takes a favorable or unfavorable 
stance with respect to some issue is likely to rate all of the subsidiary aspects of that 
issue in a manner consistent with his over-all position. In particular, people who are 
favorable to the ExTFP as a whole would probably be partial to all its parts. We have 
already seen evidence of such a tendency in the responses of the program directors. So 
a certain degree of correlation must be expected between any pair of items from the same 
questionnaire as a simple function of this bias. However, such a bias cannot be invoked 
as an explanation when items from different questionnaires correlate with one another; when 
two different sources agree in their ratings of an institution on some dimensions, the 
bias of either source alone cannot be invoked as an explanation. For this reason, special 
[ attention must be given to the correlations between judgments that were made by different 
sources . 

r C. Correlations Among Measures of Effectiveness and Satisfaction 

[ It has already become evident that faculty members, fellows, and evaluators 

I all showed favorable opinions of the effectiveness of the ExTFP. The question at hand 
is whether an institution that was ranked high on one measure also received a high ranking 
on another. For purposes of this presentation, measures of satisfaction and effectiveness 
will be combined in one correlation matrix owing tc> the fact, as we shall see, that the 
two kinds of measures had very high correlations with one another. 



Table 10 presents the pattern of correlations among eight measures of effectiveness 
and satisfaction. It is apparent that there were consistently high positive correlations 
among such ratings when they were given by the same source. Thus, the average correlation 
among the faculty items was + .55and the average correlation among the three fellow 
items was + .72. High correlations were also obtained between judgments by the fellows 
and those by evaluators, the average correlation being + .55. As to correlations between 
faculty judgments and those by the other two sources, however, only faculty ratings of 
whether the program met the fellows' educational needs correlated with all of the items 
from the other sources. In addition, faculty judgments of whether the fellows would become 
better teachers correlated with fellows' judgments of effectiveness and satisfaction but 
not with evaluators' judgments. Faculty members' own reactions to the program and their 
ratings of its value to the fellows did not correlate significantly with any of the ratings 
of satisfaction and effectiveness that fellows or evaluators made. 



Table 10. Correlations among Measures of Effectiveness and Satisfaction 







Faculty 


Fellows 


Ev. 


Source 


Item 


16 18 20a 26 


23 24 31 


19 


Faculty 


16. Reaction to ExTFP 


~ .43 .63 .69 


.26 .31 .34 


.24 




18. Did fellows become better teacher? 


.43 — .43 .62 


.43 .46 . 55 


.14 




20a. ExTFP valuable for fellows? 


.63 .43 — .49 


.02 .01 .17 


.12 




26. ExTFP meet fellowi^ needs? 


.69 .62 .49 — 


.41 .58 .61 


.44 


Fellows 


23. ExTFP meet your major need? 


.26 .43 .02 .41 


— .61 .74 


.40 




24. Reaction to ExTFP 


.31 .46 .01 .58 


.61 -- .81 


.60 




31 . ExTFP help your teaching ? 


.34 . 55 .17 .61 


.74 .81 -- 


.65 


Evaluator 


19. ExTFP meet fellows' needs? 


.24 .14 .12 .44 




.40 .60 .65 


— 



The fact that the fellows' judgments of effectiveness correlated with those by 
the evaluators and also with faculty members' estimates of whether the ExTFP met the 
fellows' educational needs is encouraging evidence of consistent, reliably-ascertained differences 
between programs in their effectiveness. Why the other faculty measures of satisfaction 
and effectiveness did not also correlate with the items from the fellows' and the evaluator 
questionnaires is not immediately clear. 



D. The Relationship Between Program Effectiveness and Program Structure 

Under this heading we will consider the correlations between program effectiveness 
and three classes of variables: (1) achievement of the en bloc approach^ solidarity, and 
morale, (2) relations among departments and institutions, and (3) the role of the director. 

In each case, we first present the correlation among variables within the set, then their 
correlations with effectiveness. 

1. The En-Bloc Approach, Solidarity, and Morale 

j. Correlation among measures. We have already learned (a) that a generally 
high level of solidarity was achieved in all of the programs but (b) some institutions had 
a consistently higher level of morale than did others. Table 11 presents the correlations 
among the different ratings of solidarity and morale. All but one of the 10 correlation 
coefficients achieved statistical significance, the mean correlation being .58. There 
was, then, remarkable agreement between sources as to which programs were characterized 
by a very high degree of solidarity and morale and which were not. It should be 
noted that the lowest correlation in this table, .30, was between the evaluators' rating 
of successful achievement of the en bloc approach and the faculty rating of student morale. 



Table 1 1 . Correlations among Measures of Solidarity and Morale 



Source 


Item 


Faculty 
48 49 


Fellows 
47 49 


Ev. 

17 


Faculty 


48. Student solidarity 


— .71 


.57 .68 


.45 




49. Student morale 


.71 - 


.57 .67 


.30 


Fellows 


47. Student solidarity 


.57 .57 


- .76 


.49 




49. Own morale 


.68 .67 


.76 - 


.56 


Evaluator 



17. En bloc successful 


.45 .30 


.49 .56 


— 



b. Relation of solidarity and morale to satisfaction and effectiveness. Table 12 
presents the correlations between measures of solidarity and morale and those of satisfaction 
and effectiveness. It is evident that the majority of these correlations were quite high, 
even when the ratings were obtained from different sources, except that fellow and 
evaluator ratings of solidarity did not correlate significantly with faculty measures of 
effectiveness. The average correlation of faculty ratings of solidarity with faculty 
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ratings of effectiveness was4.50, with fellows* ratings of effectiveness, 4.46, and with 
the evaluator rating of effectiveness, 4.47. The average correlation of fellows' ratings of 
solidarity with their own ratings of effectiveness was4.47, with the evaluator rating of 
effectiveness, 4 .42, and with faculty ratings of effectiveness, 4.28. Finally, the evaluator 
rating of achievement of the en bloc approach correlated 4 .71 with the evaluator measure 
of effectiveness, had an average correlation of 4. 41 with fellows' measures of effectiveness, 
arxl an average correlation of only 4. 18 with faculty ratings of effectiveness. Whatever 
the causal factors that may be involved in these correlations, it is clear that the achievement 
of the en bloc approach and of solidarity and morale was associated with program effectiveness, 
especially as viewed by fellows and evaluators. 



Table 12. Correlations of Solidarity and Morale with Program Effectiveness 



Source 


Item 


Faculty 

16 18 20a 26 


Fellows 
23 24 31 


Ev. 

19 


Faculty 


48. Student solidarity 


.43 .42 .25 .44 


.35 .60 .58 


.57 




49. Student morale 


.62 .62 .62 .63 


.22 .47 .51 


.37 


Fellows 


47. Student solidarity 


.27 .37 .22 .32 


.53 .64 . 55 


.44 




49. Own morale 


.30 .25 .22 .30 


.13 .57 .41 


.40 


Evaluator 


17. En bloc successful 


.28 .08 .13 .21 


.29 .51 .44 


.71 



2. Relatio ns among Departments and Institutions . 

a. Correlations among measures. Under this heading will be considered evaluator 
and fellow ratings of cooperation between departments, and evaluator judgments 
of whether the institution had established relationships with the local school systems. 

It will be remembered that respondents reported great variability among programs in 
the extent of cooperation between subject-matter and teacher-education departments, 
and in the amount of cooperation with local school systems. Table 13 presents the 
correlations among the three measures of cooperation. None of these correlations is 
higher than4.21 . This independence of one set of responses from another reveals, first, 
that according to evaluators* reports, whether subject-matter and teacher-education 
departments cooperated had no bearing upon whether cooperation was established between 
the institution and local school systems. Second, the low correlation means that 
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judgments of departmental cooperation made by fellows were either based upon different 
criteria from those used by evaluators or that the two groups used their criteria differently 
in these judgments. 



Table 13. Correlations among Measures of Departmental and Institutional Cooperation 



Source 


Item 


Fellows 

38 


EvaTuator 
3 4 


Fellows 


38. Departments cooperate 


— 


.15 .21 


Evaluator 


3. Departments cooperate 


.15 


.18 




4. Coop, with local schools 


.21 


.18 



b . Correlations of measures of cooperation with program effectiveness. 

Despite the lack of correlation among these measures of cooperation, we see in Table 14 
that both fellows' and evaluators' assessments of departmental cooperation were correlated 
significantly with ratings of effectiveness made by fellows and evaluators; however, 
they were consistently uncorrelated with faculty ratings of program effectiveness. Thus, 
although the fellows may have used different criteria from evaluators in judging departmental 
cooperation, by either criterion, programs that were rated as having a relatively high 
degree of cooperation among departments were more likely than not to be adjudged 
effective. The degree of cooperation with local school systems, as reported by evaluators, 
was substantially unrelated to any measures of effectiveness. 



Table 14. Correlations of Departmental Cooperation with Program Effectiveness 



Source 


Item 


Faculty 

16 18 20a 26 


Fellows Ev. 

23 24 31 19 


Fellows 


38. Departments cooperate 


.12 .21 -.11 .35 


.56 .70 .60 . 54 


Evaluatoi 


3. Departments cooperate 

4. Coop, with local schools 


-.04 -.20 -.12 .06 

-.10 -.10 -.11 .27 


.40 .32 .47 .44 

.14 .26 .31 .35 




- 36 - 



E. The Relationship Between Program Effectiveness and Program Strategy 

The reader will recall that the general topic of program strategy subsumed ratings 
of the utilization of fellows' background, the amount of competitiveness and the work load, 
the involvement of the faculty, and the extent of innovation in the programs. The 
correlation of each of these variables with effectiveness will be discussed in turn. 

1 . Utilization of Fellows' Background 



a. Correlation among measures. In their judgments of whether the program's 
organization tooUnlo account the ext^sive experience of fellows, evaluation te ns 
were distinct from the other three groups in saying that the majority did not. Though 
they disagreed with the fellows in the extent to which they felt that fellows backgrounds 
were utilized, the evaluation teams clearly arrayed institutions on this variable in about 
the same order as did the fellows, for the correlation between ratings from these two 
sources was4.45. This correlation, in fact, was the largest correlation iri the matrix 
(Table 15); the only other significant correlation is that between fellows judgments on his 
variable and faculty statements that the program was modified to take advantage of 
fellows' experience. 



Table 15. Correlations among Measures of Whether the Programs Utilized Fellows 
Backgrounds 



Source 


Item 


Faculty 

45 46 


Fellows 

42 


Ev. 

16 


Faculty 


45. Effort to use experience 

46. Modify prog, for experience 


.06 

.06 


.05 

.38 


.22 

.07 


Fellows 


42. Build on experience 


.05 .38 


— 


.45 


Evaluator 


16. Take acct. of experience 
— 


.22 .07 


.45 


— 



b . Correlations of utilization of participants' backgrounds with program 
effectiveness and solidarity. As is shown in lable 16, faculty judpents of whet..er^t„e 
prog ram u ti I i zed f e I lows ' exper i ences were not significantly related to any measure of 
effectiveness; faculty statements that the program was modified to take advantage c, the 
fellows' experience were generally related to the faculty's own estimates of effectiveness. 
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jut not to those by fellows and by evaluators. On the other hand, evaluators' and fellows' 

judgments on this same measure were significantly related to every fellow and evaluator 

measure of effectiveness and also to faculty judgments that the program met the educational 

needs of the fellows. We may conclude, then, that the programs whose organization built 

best on the backgrounds of the fellows according to the i nterpretation of the evaluators 

and the fellows (and we do not know tlie criteria on which these sources based their Interpretations) 

were also adjudged to be more effective. 



Table 16. Correlations of Utilization of Participants' Backgrounds and Program Effectiveness 



Source 


item 


Faculty 

16 20a 26 18 


Fellov/s 
23 24 31 


Ev. 

19 


Faculty 


45 . Effort to use exp . 

46 , Modify prog . for exp . 


-.19 .04 -.13 -.25 

.54 .31 .53 .27 


-.22 -.06 -.10 
.12 .21 .23 


.06 

.05 


Fellows 


42. Build on experience 


.27 .16 .51 .53 


.48 .71 .76 


.47 


Evaluator 


16. Take acct, of exp. 


.16 .04 .44 .03 


.34 .60 .60 


.71 



2 . Comp etitiveness and Work Load 

g. Correlations among measures. The correlations reported in Table 17 lend 
I empirical support to our earlier conclusion that the faculty judgments about student work 
i load represented a positive statement about the fellows, not an objective assessment of 
: the amount of work they were required to do: every correlation of the faculty judgments 
: on this item with those of fellows or evaluators was negative. On the other hand, fellows' 
i judgments of their work load correlated positively and significantly with those of the 
^ evaluators. It should be noted, in addition, that neither of these last two measures -- 
; fellows' and evaluators' estimates of the work load — correlated significantly with fellows' 
: statements about the level of competitiveness in their programs; clearly, fellows could 
i believe they were overworked in either a competitive or a non-competitive atmosphere. 
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Table 17. Correlations among Measures of Competitiveness and Work Load 







Faculty 


Fellows 




Ev. 


Source 


Item 


35 


26 


29 


14 


Faculty 


35. Did students work hard 


— 


-.39 


.31 


-.26 


Fellows 


26. Work Load 


-.39 




.28 


.54 




29. Level of competition 


-.31 


.28 




.01 


Evaluator 


14. Work Load 


-.26 


.54 


.01 


— 



b. Relationship of competitiveness and work load to program effectiveness. 

Further evidence for our conclusion that faculty ratings of the amount of work the fellows did 
actually represent favorable judgments of their performance is given in row 1 of Table 18. 
This measure correlated positively and significantly with every faculty rating of program 
effectiveness and with two of the three effectiveness ratings made by fellows, 

A different pattern held for Tcl^'^ws' ratings of their work load. These judgments 
showed a high negative correlation with fellows' opinions that the program was stimulating 
and interesting, and moderate negative correlations with the other ratings of program 
effectiveness by the fellows and evaluators as well as with faculty judgments of whether 
the program produced better teachers. Evaluator ratings of work load also showed negative 
correlations with fellow and evaluator ratings of effectiveness. As to the level of 
competitiveness in the program, while the correlations of this measure with judgments of 
effectiveness and satisfaction were consistently negative, they barely achieved statistical 
significance in only two cases. In short, programs in which fellows and evaluators reported 
that the work load was excessive tended also to be programs which received low marks 
for effectiveness cmd satisfaction, but a program that was viewed as competitive wos not 
necessarily ineffective. 



Table 18. Correlations of Competitiveness and Work Load with Program Effectiveness 



Source 


Item 


Faculty 

16 20a 26 18 


Fellows 

23 24 31 


Ev, 

io 
1 / 


Faculty 


35. Work hard? 


.49 


.41 .63 .62 


.22 .51 .46 


.24 


Fellows 


26. Fellows' work load 


-.34 


.03 -.21 -.43 


-.35 -.72 -.30 


-.32 




29. Competitiveness 


-.17 


-.28 -.24 -.39 


.01 -.39 -.30 


.04 


Evaluator 


14. Fellows' work load 


-.26 


CM 

• 

1 

O 

CO 

• 

1 

00 

o 

• 


-.24 -.44 -.35 


-.38 
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3. Involvement of the Faculty 



a. Correlations among measures. There were five items that bore on the faculty s 
involvement in the ExTFP: two of these“^ked the faculty whether the ExTFP had been 
challenging and satisfying, one asked evaluators if the faculty had been challenged arid 
stimulated by the ExTFP, and Jwo asked fellows whether the faculty had been accessibl 
and helpful. The correlations among these measures, presented in Table 19 show that ^ 
almost the only significant correlations are between measures from the same questionnaire. 
Thus, institutes in which the faculty said they were challenged were also those in which 
the faculty found the teaching satisfying; schocis in which fellows '^eported the 
accessible were schools In which fellows said the faculty were helpful. The only siginificant 
correlation between items from different questionnaires was between fellows reports ot 
faculty helpfulness and evaluator ratings of faculty stimulation. 



Table 19. Correlations among Measures of Faculty Involvement 



Source | 


Item 


Faculty 
24 25 


Fellows 
41a 41b 


Ev. 

9a 


Faculty 


24. Was teaching challenging? 

25. Was teaching satisfying? 


.56 

.56 


.19 .30 

.10 .28 


.33 

.14 


Fellows 


41a. Were faculty accessible ? 
41b. Were faculty helpful ? 


.19 .10 

.30 .28 


.87 

.87 


.24 

.41 


Evaluator 


9a. Was staff challenged? 


.33 .14 


.24 .41 


— 



b. Correlations between faculty involvement and effectiveness. The pattern 
of correlations between faculty involvement and effectiveness, presented in Tabie 20, 
is not easy to understand. One of the faculty measures, statements about whether the teaching 
experience was satisfying, correlated with virtually every measure of effectiveness and 
satisfaction - perhaps because it might, itself, be called a measure of satisfaction 
Evaluators' judgments of whether the faculty was challenged were correlated w^ith fellow 
and evaluator, but not faculty, measures of effectiveness. Fellow ratings of ti.e .acu.ty s 
helpfulness and accessibility correlated significantly only with their own judgments of 

effectiveness. 
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Table 20. Correlations of Faculty Involvement with Program Effectiveness 



Source 




Measures of Program Effectiveness 




Item 


Faculty 

16 20a 26 18 


Fellows 
23 24 31 


Ev . 
19 


Faculty 


24. Teaching cha II . 

25. Teaching satisf . 


.42 .09 .31 .28 

.75 .27 .55 .46 


.26 .25 .26 
.31 .45 .50 


.18 

.30 


Fellows 


41a. Fac . accessible 
41b . Fac . helpful 


-.01 -.25 -.01 .23 

.14 -.11 .23 .38 


.58 .40 .29 
.65 .66 .49 


.15 

.32 


Evaluator 


9a. Fac. challenged 


1 .16 .01 .24 .08 


.44 .51 .45 


.68 



4. Innovativeness 

As we have seen, on the two measures of innovativeness, neither faculty members 
nor evaluators reported any appreciable degree of innovation. Nor did measures of 

innovation from the two sources vary jointly: the correlation between them was .1 . 



On the other hand, os may be seen in Table 21, the faculty estimate of i;"°:«>«veness 
was related to every measure of program effectiveness; the average correlation of variable 
with faculty ratings of effectiveness vwis .55, with fellow ratings of effectiveness, .48, and 
with the e,^luator rating of effectiveness, .38 (Table 21). Evaluator ratings of innovativeness, 
on the other hand, were significantly related only to the evaluator measure of 
and to faculty judgments of whether the fellows became better teachers as a result o their 
experience. We see, then, that by the evaluators’ standards of innovation our 
generalization holds up: programs could be effective whether they 

or substantially traditional . It should be noted that this question asked faculty members 
whether they had observed innovative teaching methods or practices; in view of this wording, 
their iudaments may have reflected inventiveness in some one teacher s performance tamer 
t ^inrativene^ in the over-all program. If so, the meaning of this variable’s correlations 
with effectiveness is considerably different from the meaning that would be carried by a 
correlation with innovativeness in the program itself. 



An alternative explanation of the correlation between these faculty ratings and 
effectiveness is that the current popular emphasis on innovation in education as serve 
to make "quality" and "innovation" in some respects synonomous for many people. Ihus 
many faculty members may have felt that if they judged their program to be successful, 

it must have been innovative, as well. 
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Table 21 . Correlations of Innovativeness with Program Effectiveness 







Measures of Effectiveness 






Faculty 


Fellows 


Ev. 


Source 


Item 


16 20a 26 18 


23 24 31 


19 


Faculty 


27. Innovativeness 


.59 .41 .79 .48 


.50 .42 . 53 


.38 


Evaluator 

1 


2. Innovativeness 


• 

• 

GO 

• 

o 


-.14 .03 — 


.44 



5 . Ef fects on i ns titutional development 

a . Correlations among measures . We have seen above that the three sources 
differed remarkably in their estimates of the Program's effect on institutional development. 
As might be expected, the pattern of correlations between variables showed the same lack 
of correspondence (Table 22): estimates by faculty members of the Program's value to the 
institution were uncorrelated with evaluators' ratings of both effects on teacher training and 
effects on departmental development. At the same time, the high correlation between the 
two evaluators' judgments shows that institutions at which evaluators felt changes were 
made in methods of teacher education also were judged to be institutions where departmental 
development was affected. This relationship, actually is partly determined by the fact that 
the "department" whose development was being affected; that is, the home department of 
the program, was in half the cases the department of education. 



Table 22. Correlations among Measures of Effects on Institutional Development 



Source 


Item 


Faculty 

20c 


Evaluator 
1 8 


"acuity 


20c. Value to institution 


— 


.16 .07 


Evaluator 


1 . Effects on teacher training 


.16 


.56 




8. Effects on department development 


.07 


.56 



b . Correlations betw een insti tutional development and effectiveness. Table 23 
presents the correlations between measures of institutional development and judgments of 
satisfaction and effectiveness. Global judgments by faculty members of the Program's 
value to the institution correlated positively with their own ratings of satisfaction and of 



program effectiveness, but were only minimally related to such ratings by other sources. 
Evaluator judgments of the Program's effects upon the host institution showed low positive 
relationships to their own and fellows' ratings of effectiveness, but were unrelated to such 
ratings by faculty members. In short, it appears that a program could be effective In 
training students without, necessarily, serving as the impetus for extensive changes In 
the pattern or organization at the host insitituion. 



Table 23. Correlations of Effects on Institutional Development with Program Effectiveness 







Measures of Program Effectiveness 






Faculty 


Fellows 


Ev. 


Source 


Item 


16 20a 26 18 


23 24 31 


19 


•acuity 


20c. Value to InstI . 


.70 .47 .47 .63 


.07 .41 .35 


.29 


Evaluator 


1 . Effects on teacher tr. 


.04 .10 .10 .20 


.29 .44 .37 


.35 




8. Effects on dept. div. 


.14 .03 .07 .22 


.14 .27 .31 


.35 



E . Summary: Correlates of Effectiveness 

Detailed comments should be made about two aspects of these results. The first 
has to do with the reliability of judgments, with whether two judgments which seem, on 
the surface, to be asking the same question do. In fact, correlate with one another; the 
second Is the consistent correlates of satisfaction and effectiveness. 

1 . The rel iability of the judgments 

In general, when a single source was asked more than one question on the same 
topic or logically related ones, the responses to those questions were positively correlated. 
Thus, faculty, fellow, or evaluator responses to one Item showed generally high correlations 
with responses by the same source to other Items whose content was similar . For example, 
institutions whose faculty members said the Program was stimulating and Interesting throughout 
were also the ones whose faculty said that the Program was of great value to the fellows, 
that It helped the fellows become better teachers, and that It met fellows' educational needs. 

However, responses by different sources to Items that were similar In content did 
not always correlate significantly. Two sets of Items — satisfaction and effectiveness, and 
solidarity and morale — showed marked consistency across all three sources; for each set of 
items, the ratings that Institutions received from faculty members paralleled those given by 
fellows and also, to a lesser extent, those given by evaluation teams. On a number of 
other factors, responses by faculty members were substantially uncorrelated with those of 
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fello//s and of evaluators. However, the responses of fellows and those of evaluation 
teams correlated virtually across the board — in ratings of satisfaction and effectiveness, of 
solidarity and morale, of fellows' work load, of whether the fellows' experience was utilized, 

I and of the extent of faculty involvement in the Program, responses from these two sources 
correlate i significantly; only in judgments of whether there was cooperation between different 
departments were fellows' judgments and those by evaluators uncorrelated. For one set of 
items — estimates of fellows' work load — faculty judgments were inversely related to those 
from the other two sources, probably because the question that was asked of faculty members 
evoked judgments of the level of fellows' motivation instead of objective assessments of 
their work load. In short, there was consistently high agreement between judgments 
by fellows and those by evaluation teams; agreement between these two sources and the 
faculty was largely restricted to two areas: (a) satisfaction and effectiveness and (b) solidarity 
and morale. 

* It is interesting to speculate about the meaning of this differential pattern of 
correlations. Since the three sources viewed the program from different perspectives, the 
pattern of correlations probably reflects such differences. It might be argued, first of all, 
that whether a program was very effective or relatively ineffective, and whether its 
fellows had very high or relatively low morale could be determined by faculty and fellows 
alike from evidence thot is public and common. People discuss with one another how much 
they have learned and how well it was presented; there are indicators of group solidarity and 
morale which almost any adult can see and identify. Ori the other hand, more subjective 
criteria are called into ploy for judging whether and ho\'/ much the fellows' prior experience 
was utilized by the program, or how deeply the faculty was involved in the program. 

Such questions are probably less frequently discussed, the bases for decision about them 
less commonly shared, than the topics of effectiveness or group solidarity. If this is true, 
judgments on these latter topics would be more likely to reflect the biasing effect of the judge's 
social position. To be more specific, it seems likely that the faculty's institutional position 
made it unlikely that they would learn much about the fellows' past experience and its 
relevance to the course material, or about fellows' judgments of whether the faculty was 
involved in the program. In the absence of explicit information, the faculty was doubtless 
likely to respond in a manner consistent with their desire that their own program be rated 
effective and "good . " 

* 

t 

These considerations would account for the lack of correlation between responses 
of fellows and faculty on issues of this second type, but not necessarily for the fact that 
judgments of evaluators paralleled those of the fellows instead of the faculty when the 
latter sources disagreed. Perhaps their discussions with the fellows exposed evaluation teams 
to information that was not available to the faculty; alternatively, perhaps evaluators 
considered the faculty to be more personally involved than fellows in the outcome of the 
evak'ition, hence more likely to be biased in their judgments. 
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Whether one of these explanations or some other one can account for 
of course requires information that is not available in the present study. Concerning tte 
reliability of judgments, we have seen that judgments made by a single source on ° s'"9le 
ss"re quite reliabk, that consistent positive correlations were found between the three 

and fellows consistently agreed with each other, but not with the faculty, on other issues. 

2 , Correlates of satisfaction and effectiveness 

Proarams that ranked high on satisfaction end rated effectiveness (a) were adjudged 
by facultylnd by fellows to have a high degree of solidarity and morale, and by evaluatio 

tLms to have been successful in achieving esprit de corpj through the^^ “ors°^xMrience 
(b) were successful in the view of fellows and of evaluators in utilizing the P^vious p 
of the fellows; and (c) were adjudged by fellows and evaluators ^ to have required 

altogether unrealistic amount of work. 

The fact that effectiveness and morale went together is not surprising; it reaffirms 
a long standing common-sense generalization. It is interesting to note 
effectiveness was correlated only with judgments by fellows ond evoluotors of whether the 
fellows' bockgrounds were utilized and of work load; judgments by faculty members o 
E lo Eues did not correlate with the same judgments by the other wo -roes, his 

suggests that the faculty and directors may often have been ^ 

oHitudes on these and other issues; indeed, spontaneous comments by evaluation teams 

rrojltiJEtitunr^ 

It seems likely that information about fellows’ attitudes might have induced the staff of 
thror^rams either to change some part of their educational structu^ and content so as 
^ f fk fpllowc' oh sections or to clarify for fellows and staff alike the reasons for 

with the fellows. 

Although fellows and evaluators did not agree as to which programs had o great 
deal of cooperation among departments and which did not, by either the 
L fellows'^riterion, programs with such cooperation were c^^^ 

r.ti is benefitted most from the Program, those institutions that either 9-up 

benefitted most were rated as most effective. Two other kinds of 

inconsistent patterns of relationships with satisfaction and "h'^f4^,i^eness 

source of the extent of faculty involvement in the program were correlated with effectiveness 

as rated by that source but not os rated by the other sources. 
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Finally, there was a significant correlation between faculty ratings of innovativeness 
and every measure of effectiveness — institutions whose faculties were impressed with 
the innovations that had been introduced through the Program were considered to be relatively 
effective by faculty, fellows, and evaluators alike; however, evaluator ratinjs of 
innovativeness were only marginally correlated with effectiveness as judged by evaluators 
and were uncorrelated with such judgments by the other two sources. We have suggested 
that faculty ratings of innovativeness may have reflected their own involvement in the program 
more than objective judgments of this phenomenon. 

We have already remarked that one should not infer causation from correlation. 
Hopefully, future studies in this series will help further to clarify the factors that account for 
differences among institutions. Our caveat against confusing correlation with causation, 
however, does not opply to the relationship between the director’s behavior and program 
effectiveness. The experience of the evaluation teams strongly suggested that an energetic, 
persuasive director with institutional power commensurate to his responsibilities could play 
a major role in assuring the effectiveness of the program. Conversely, a promising program 
was sometimes rendered less effective by an inept director, one with insufficient time to give 
to his duties, or one denied the power to institute and carry through both general policies 
and the specific procedures necessary to implement those policies. 
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